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Abstract. 

Astro-WIS EPl dVriend et all2012h is the Astronomical Wide-field Imaging System 



for Europe dValentiin et al.ll2007l) . It is a scientific information system which consists 
of hardware and software federated over about a dozen institutes throughout Europe. 
It has been developed to exploit the ever increasing avalanche of data produced by 
astronomical surveys and data intensive scientific experiments in general. 

The demo explains the architecture of the Astro-WISE information system and 
shows the use of Astro-WISE interfaces. Wide-field astronomical images are derived 
from the raw image to the final catalog according to the user's request. The demo is 
based on the standard Astro-WISE guided tour, which can be accessed from the Astro- 
WISE website. 

The typical Astro-WISE data processing chain is shown, which can be used for 
data handling for a variety of different instruments, currently 14, including Omega- 
CAM, MegaCam, WFI, WFC, ACS/HST, etcfl 



1. Typical Data Processing Chain 

Astro-WISE was originally designed to handle the large datasets fr om the OmegaCAM 
instrument such as the KiDS survey dVerdoes Kleijn et al.ll2012h . The backbone of 



Astro-WISE is set by the way it captures all science products obtained in survey oper- 
ations in an object-oriented data model. Figure Q] shows the main astronomical classes 
in the Astro-WISE environment, the basic elements of the data processing chain. Each 
block is a class and each instance of a class is called a Target. The class incorporates a 
method to derive the data of the Target from other objects, called its dependencies. 

The user has an ability to combine recipes in a pipeline to process the data by 
directly requesting the final required data product he or she is interested in. All pro- 
cessing parameters, along with the full data lineage, are saved in the metadata through 
the persistence of all objects. In the demo the user goes from a RawScienceFrame (raw 
frame observed by the VST and ingested in Astro-WISE) to a SourceList (the cata- 
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Figure 1. A target diagram: slightly simplified object model that is a view of 
the dependencies of "targets" to the ocean of raw observational data of astronomical 
wide-field imaging. The arrows indicate the backward chaining to the raw data, 
not the progression through any processing pipeline. The colors provide a visual 
grouping of similar types of data products. 



log produced from reduced, regridded and coadded imagesfl using the Astro-WISE 
infrastructure through the web services described below. 

An Astro-WISE node is the building element of Astro-WISE infrastructure. It con- 
sists of data storage element (dataserver, which stores all the files with images), meta- 
data database (RDBMS, which keeps metadata including links between data items), 

computing elements (Distributed Processing Unit) and a number of interfaces and ser- 

vices w hich allow to the user to browse and process data stored in the system (see lBegeman et al 
(120101) for more technical details). 



2. Services and Interfaces 

The main language for the system is Python, but each user can develop her/his own 
application or use an existing application which can be wrapped into Python. Usu- 
ally, users develop pipelines or workflows using existing "blocks" with the help of pre- 
defined Python libraries and classes. The user can also change an existing data model 
if necessary or implement a new one. 

The Command Line Interface of Astro-WISE, AWE (Astro-WISE Environment), 
can be installed on a site without any other components of Astro-WISE (data server 
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and metadata database). Basically the AWE prompt is a link to a local Python installation 
with the Astro-WISE libraries and environments. 

Apart from the AWE prompt, Astro-WISE supports a range of web interfaces. This 
makes it possible for a user to work with data stored in Astro-WISE without the AWE 
prompt using a web browser only. The web interfaces are divided into two types: data 
browsing/exploration and data processing/qualification. The first group includes: 

• dbviewei0- the metadata database interface which allows browsing and querying 
all attributes of all persistent objects stored in the system, 

• Go-WIS^] - allows querying on a limited subset of attributes of the data model 
(coordinate range and object name), and provides results of all projects, 

• image cut out servicd3 and color image makeiQ - these two services are for the 
astronomical image data type and allows creation of cut-outs of an image or the 
creation of a pseudo-color RGB image from three different images of the same 
part of sky, 

• skymarj^- exploration tool of the Astro-WISE system using the GoogleSky in- 
terface. 

Data processing / qualification interfaces are: 

• target processoi0 - the main web tool to process the data in Astro-WISE. This 
web interface allows the user to go through pre-defined processing chains sub- 
mitting jobs on the Astro-WISE computing resources with the ability to select the 
computing node of Astro-WISE. The Target Processor allows for implicit collab- 
oration by indicating that objects can be reprocessed when another scientist has 
created improved versions of the objects that it depends on, 

• quality serviced - allows the estimation the quality of the data processing and 
set a flag highlighting the quality of the data, 

• CalTS03 - web interface for qualifying and time stamping calibration data. 

The exact set of web interfaces depends on the data model implemented in the 
system. The web interfaces described above are for the optical image data processing 
and follow the requirements for this particular data model and data processing chain. 
Astro-WISE allows the implementation of new web interfaces for the data model and 
for data processing defined by the user. The developer of the new web interface will 
use pre-defined classes and libraries of Astro-WISE to create it. 
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3. Data publishing and External Data 

Data access interfaces from the Virtual Observatory exist as separate services^ that en- 
ables browsing the metadata database and retrieving the data from dataservers. Astro- 
WISE supports the Simple Image Access Protocol for images and ConeSearch for 
sources. Each data entity in Astro-WISE has a persistent attribute which shows scope 
of visibility for this entity, which allows the creator of the object to specify with whom 
to share the object. 

4. Conclusion 

The demo of Astro-WISE is based on the Astro-WISE guided tour and tutorial, which 
can be found on the Astro-WISE website. It shows how the request driven way of pro- 
cessing and full data lineage gives Astro-WISE the power to handle the large datasets 
produced by surveys such as KIDS. The user can apply standard data processing using 
target processing or can develop his/her own recipe for the data processing using the 
Astro-WISE pipeline as the building blocks. 
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